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Within the framework of a general model of 
communication, a model of language communication has been developed 
and applied to the perception of cohesion. To measure students* 
perception of textual cohesion, a pilot study in the United Kingdom 
built "noise" into texts by deleting parts of each texts. Subjects, 
59 nonremedial students whose first language was either Italian or 
Punjabi, were given two test booklets in English, one fiction and one 
nonfiction. Each booklet contained 65 text deletions. Using an 
accepted guideline of miscue analysis, results did not indicate 
serious weaknesses in the students* reading at the points measured. 
Examining the measure of consensus displayed by students* responses 
leads to the intuitive conclusion that in these cases consensus 
measure is more important than exact match score. The results confirm 
that this type of analysis can provide potentially useful insights 
into readers* text processing strategies. (JL) 
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The purpose of this article is to describe how students' perception 
of textual cohesion may be measured. • To do this it is necessary first to 
consider perception of textual cohesion within the general model of 
communication for this model provides the theoretical underpinning for the 
technique advocated. Next, a method of quantifying the signals readers 
receive (that is, readers' perception of signals transmitted by writers) 
is outlined. The method of measurement is then illustrated using as subjects 
students for whom English is a second language. 

A model of language oorrmmioation 

In any consideration of reading comprehension there is a text (cu:id 
by implication a writer of the text) and there is a reader. It is for 
convenience only, as Chapman (1981b) has observed, that text and reader 
are sometimes considered separately. Any model of reading comprehension 
or Icinguage communication must take into account, at the very least, the 
interaction of writer, text and reader. The writer, when he encodes meanings 
in the form nf text, selects successively from a set of possible alternatives? 
similarly, the reader in decoding text selects from the same set of 
alternatives. If there were perfect communication, the ir.eanings derived 
by the reader would be identical to those intended by the writer. However, 
communication is seldom perfect and, as noted by Lyons (1977) , any model 
must allow for this theoretically. 

The generalised communication model (see Figure 1) has been widely 
•J* , 
applied to biological, psychological, social and other systems since it was 

developed by Shannon and Weaver (1949) to deal with signal transmission. 
The adaptation of this model to language communication is described by 
Anderson (1976). Anderson's model (see Figure 2) has four components: the 
source system (in present discussion , the writer) , the message system (the text) , 
the receiver system (the reader) , and the noise system (to allow for the possibility 
of misunderstanding between reader and writer) . 
Applioation of the model to peroeption of cohesion 

The previous paper (Chapman 1982) has described Halliday and Hasan's 
concept of cohesion - the semantic linking mechanisms between sentences 
that make a text a text (Halliday and Hasan 1976 p. 13). In the longitudinal 
study being conducted from The Open University into post-primary students' 
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perception of textual cohesion (Chapman 1980, 1981a), the between- sentence 
cx>hesive ties, in a series of passages representative of reading materials 
encountered in schools, were identified according to the categories of 
Halliday and Hasan, The particular use of • ohesive ties in texts is part 
of the choice exercised by writers in their encoding of meaning. To 
measure students' perception of textual cohesion, one end of each tie 
identified for inclusion in the study was deleted. In other words, noise 
was deliberately built into the message system or text by deleting parts 
of -each text and the reader's task was then to restore the deleted text 
elements and reconstruct the texts. 

According to the language communication model depicted in Figure 2, 
the transmission and receiving of messages are seen as essentially coding 
operations. The writer in encoding the message is influenced by his 
encoding habits, his associations, attitudes and values and these habits, 
associations, attitudes and values are reflected in the coded message or 
text. Noise in the form of breaks in the pattern of ties between sentences 
forcibly interrupts the message. In order to decode or recover the message, 
the reader utilizing all his relevant past experiences (which include his 
previous learning, his associations, attitudes and values) needs, among 
other things, to locate the non-deleted end of the tie embedded in the 
text and make a prediction based upon it and other cues in the text. All 
this is in accord with the psycholinguistic view of reading which sees the 
obtaining of meaning from written material as involving strategies of . . 
predicting and testing and then confirming or correcting (Goodman 1972; 
Clay 1976). Now to the extent that readers are successful, in restoring 
the missing ends of each tie and thus completing the semantic links, they 
are said to have perceived the cohesive ties. 

There are several advantages of the model of language communication 
for measuring perception of textual cohesion. First, students employ, in 
the task required of them, the same reading strategies as those employed 
in their other reading. That is, in making predictions, testing these, 
and then confirming or correcting predictions made, readers make use of 
the graphophonic, syntactic and semantic cue systems in language (Goodman 
1969, 1973; Smith 1973, 1978). Choice points are set up where the deletions 
occur to provide "a window on the reading process" which otherwise in silent 
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reading cannot be directly observed. While these choice points may 
interrupt the reading process, they are not unlike the same choice points 
faced by the writer in coding the message and, indeed, not unlike the 
pauses and starts encountered in regular reading. Second, the model permits 
well established methods of analysis from diverse other fields to be 
applied, provided due care is taken not to violate basic assumptions. 
Below, for example, some of the concepts from information theory and some 
of the theoretical insights from miscue analysis are brought to b^ar on 
the quantification of perception of textual cohesion. Third, the model 
provides a framework for describing what takes place in text comprehension. 
Fourth, and most importantly, the model is useful to the extent that it 
provides hypotheses or explains phenomena. 

Quantifying peroeption of textual cohesion 

In a slightly different context Osgood (1959) summed up how a model 

such as that in Figure 2 may be used to measure readers' ^perception of 

signals transmitted by a writer: 

The human source that produces the message has a very complex 
and hierarchical system of language-encoding habits. .. Similarly , 
the receivers of his message have equally complex and 
hierarchical systems of language habits. To the degree that 
these complex habit systems of source and receiver correspond, 
one should be able to substitute for the other, that is, 
should be able to complete his messages (p. 79) . 

The correspondence of the language-encoding habits of writer and the 
language-decoding habits of reader provides an operational definition of 
perception of textual cohesion in the special case where one end of a 
cohesive tie selected by a writer is omitted and the reader's task is to 
make the most likely replacement. The measurement of this correspondence 
is also quite straight- forward. One way is to use the writer's chosen word 
or phrase as the sole criterion and match the reader's replacement against 
this: an exact match is given credit, a mismatch is not. Alternatively, 
the procedures of miscue analysis may be adapted and each reader's 
replacement judged against the writer's original choice in terms of 
grammatical and semantic acceptability. Replacements are then evaluated 
in terms of whether both language and meaning (over the span of the tie or 
over the whole passage) are acceptable, are not acceptable, or are partially 
acceptable. 
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The ^^^thods of measurement just described above may be used for 
individual readers or for groups of readers. The essential characteristic 
of the two measures is that each relies on a criterion (the writer's actual 
choice of words) against which readers' chosen replacements are judged for 
correctness. In the one case, exact matches are scored correct; in the 
other, partial scoring is used depending on the degree of language and 
meaning retention. 

There is a third type of measure which may be compute?d where there is 
a group of readers, each of whom attempts to replace the ends of cohesive 
ties in a given text. This form of measurement, however, differs from 
those described in that there is no notion of a single correct response; 
rather all responses elicited by the group of readers are considered and 
weights applied according to the number of times each response is chosen. 
The derivation of two such measures - estimated entropy and consensus - are 
described elsewhere (Anderson 1981) and owe their origins to the mathematical 
theory of communication. The measure of consensus, as its name implies, 
is an index of the degree of agreement among respondents. In the two 
limiting cases, where there is complete agreement among respondents and 
where there is no agreement at all among respondents, the measure of consensus 
is unity and zero respectively. Where the responses cluster, the measure 
takes on an intermediate value; the greater the cluster, the higher the 
measure of consensus. Entropy is the complementary quantity and may thus 
be thought of as an index of uncertainty. For instance, where consensus 
is unity, entropy is zero. The terms estimated entropy and (estimated) 
consensus are used because the measures are estimates, being based on 
particular samples of respondents. 

Data collection 

In parallel to the large-scale longitudinal project noted above into 
students' perception of textual cohesion (Chapman 1980, 1981a), a small 
exploratory study was undertaken in a small number of schools with s\±>jects 
for whom English was a second language. This study was funded by the 
Research Committee of the Faculty of Educational Studies at The Open 
University and the assistance afforded in administering the test booklets 
is duly acknowledged. Below a small part of the data is analysed to 



illustrate how students' perception of textual cohesion, as described, is 
measured. 

The five schools participating in the study were all located in 
Bedfordshire, within easy access of The Open University. The schools 
were approached because of the known groups of students attending who 
spoke either Italian or Punjabi as their first language. The purpose of 
the study was not to make a comparison across language groups (the numbers 
were in any event too small) but to try out the test booklets with subjects 
for whom English was not the native language. The only requirement was 
that the target group should be children who were coping in non-remedial 
classes. It transpired that some children did not have adequate English 
even to attempt the test». which was unkind to the children concerned and 
highlighted the differing interpretations of the term remedial . 

It is worth noting the difficulties encountered in selecting the two 
groups of children with Italian or Punjal)i as their first language: 

1. No records were kept by the schools of the children's mother tongue, 
nor were teachers generally aware of what particular ihdian or Italian 
language was spoken at home, nor to what degree it was spoken. 

2. Some children were unaware of what the language they spoke at home 
was called. 

3. The only criterion left in many cases was the child's name. This was 
insufficient evidence upon which to differentiate between Italian 
and Sicilian and was an area fraught with difficulty in the case of 
Indian names where many names cross language bo\indaries and where, in 
any case, children are often quite unfamiliar with and \inable either 
to write or pronounce the names considered to be their family names. 

4. In order to ascertain whether P\injabi/Italian was the children's first 
or second language, it was necessary to question them individually 
about what language they spoke at home, thought in and used among 
friends of the same nationality. This was complicated by the feeling 
that there was a certain amo\int of snobbery attached to speaking 
more English. In some cases it became apparent that the child spoke 
the mother tongue to one or both parents while using the English among 
peers both at home and at school. 
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Two test booklets, one with fiction and the other with non-fiction 
texts, were administered, each booklet containing 65 text deletions. Of 
these, 60 items were classified as reference, substitution, conjunction 
or lexical cohesion (ellipsis was not included) and there were 5 non- 
cohesive items. Two other test instruments were administered, a standardised 
reading test and a measure of non-verbal ability, though analysis involving 
these tests is not discussed here. The testing was carried out over two v 
separate days. Of the 69 subjects tested, full test results were available 
for 59 (24 Italian and 35 Punjabi) subjects and it is the analysis of these 
that is reported below. 

Evaluating reading responses 

The purpose of this section is to illustrate the method of measurement 
by considering a small selection of the 130 items in the fiction and non- 
fiction test booklets. The initial examples below are from Halliday and 
Hasan's (1976) reference category of cohesion (sub-category: personal 
reference) in which the identity of relation is signalled by the use of 
personal pronouns, possessive adjectives cr possessive pronouns. 

Tables 1 and 2 show two segments, one from a fiction and the other a 
non- fiction text, in which the respective writers use the personal reference 
[it] anaphorically, to refer back in the fiction text to house and in the 
non-fiction text to spider . For each segment the responses given by the 
59 readers are tabulated in terms of grammatical and semantic acceptability. 
The judgment of grammatical acceptability can only be made over the sentence 
in which the deletion occurs; the judment of semantic acceptability, on 
the other hand, covers not only the span of the cohesive tie but surrounding 
sentences as well. The 'partially acceptable' category is used where a 
response is neither clearly acceptable nor clearly unacceptable. 

Tables 3 and 4 show two further segments in which the writers' use of 
the presupposing item [they] refers back in the fiction text to the various 
children mentioned and in the non-fiction text to labradors ( the children 
and labradors being the presupposed items respectively) . 
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TABLE 1 



Responses of 59 Readers to Personal Reference Item [it] in fiction 
text 



The Lion 


• the Witch and the Wardrobe 




..."It is a 

would like — 


lovely place, my house," said the Queen. 

ThfiT-fi are whole rooms full of 


"I am sure you 
Turkish Delight, 


and what 


's more, I have no children of my own. 


— «^ ' — 


Response 


No. 


Grammatically Acceptable 
Yes Partially No 


Semant i cal Iv Acceptcible 
Yes Partially 


it 


35 


/ 


/ 


to 


6 


/ 


/ 


this 


2 


/ 


/ 


thein 


2 


/ 


/ 


Turkish 


2 


/ 


/ 


Delight 


1 


/ 


/ 


some 


1 


/ 


/ 


stay 


1 


/ 


/ 


pig 


1 


/ 


/ 


all 


1 


/ 


/ 


and 


1 


/ 


/ 




6 


/ 


/ 


Percentage 


100 


80 3 17 


73 10 17 



Exact match 



Consensus 



59% 
58% 
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TABLE 2 



Responses of 59 Readers to PerMnal Reference Item [it] in non- 
fiction text 



The Spider 

...Then an insect flies into the web. The insect struggles and the web 
shakes. The spider's 'telephone' shakes too. The spider comes running 
down. binds the insect with silk. . . 



Response 

- 


No 


Grammatically Aaceptable 
Yes Partially No 


Semantically Acceptable 
Yes Partially No 


it 


6 


/ 


/ 


he 


6 


/ 


, / 


and 


15 


/ 


/ 


then 


5 


/ 


/ 


they 


1 


/ 


/ 


the 


13 


/ 


/ 


to 


4 


/ 


/ 


but 


2 


/ 


/ 


down 


1 


/ 


/ 


like 


1 


/ 


/ 


telephone 


1 


/ 


/ 




4 


/ 


/ 


Percentage 


100 


20 34 46 


54 2 44 



Exact match 
Consensus 



10% 
46% 
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TABLE 3 



Responses of 59 Readers to Personal Reference Item [they] in 
fiction text 



The Children Wait for Macfio Man 

..."Let's all meet at twelve o'clock on Rosie's cellar door", said Pudgy. 

"Twelve o'clock sharp!" said Sal. 

"Sharp!" they all agreed. 

And then went home. 



Response 


No 


Grammatically Acceptable 
Yes Partially No 


Semantically Acceptable 
Yes Partially No 


they 


49 


/ 






/ 


all 


2 


/ 






/ 


we 


3 


/ 






/ 


he 


1 


/ 






/ 


she 


1 


/ 






/ 


went 


1 






/ 


/ 




2 






/ 


/ 


Percentage 


100 


95 




5 


86 8 5 



Exact match 
Consensus 



83% 
81% 
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TABLE 4 



Responses 


of 59 


Readers 


to Personal 


Reference 


Item [they] in 


non- fiction 


text 








LJyJLL/ J. i^KAL/J. llf^ L'J. C/t^x O 










L2j3radors are famous 
breeds for training 
a« 1 as 


for their obedience, and they are one of the best 
as Guide Dogs for the Blind. Not all kinds learn 
do, of course, but every dog should be taught 


to obey. 














Response 


No 




Grammatically Acceptabl3 
Yes Partially No 


Semantically Acceptable 
Yes Partially No 


they 


17 




/ 






/ 


others 


3 




/ 






/ 


labradors 


1 




/ 






/ 


labrador 


1 






/ 




/ 


some 


2 




/ 








dogs 


2 




/ 






/ 


you 


6 




/ 






/ 


your 


1 








/ 


/ 


well 


1 








/ 


/ 


to 


1 








/ 


/ 


tan 


1 








/ 


/ 


it 


1 








/ 


/ 


we 


1 




/ • 






/ 


put 


1 








/ 


/ 


blind 


1 








/ 


/ 


if 


1 
15 








/ 
/ 


/ 
/ 


the 


3 








/ 


/ 


percentage 


100 




52 


2 


46 


41 3 56 



The analysis of readers' responses in Tables 1 - 4 is illustrative. 
Four items only are analysed but certain aspects of the texts, for these 
subjects, is highlighted. The nature of the task required readers to 
focus on the underlying threads of continuity within texts, Halliday and 
Hasan's (1976) text-forming relations. Although the between-sentence ties 
in the analyses presented here may all be completed with the personal 
pronoun, examination of the responses shows, as expected, that some factors 
influencing readers are text dependent (hence responses like Turkish and 
Delight ) , while other factors result from interaction of text and reader 
(and hence seemingly idiosyncratic responses like pig ) * If the author's 
choice of a word or phrase to form each tie is taken as criterion of 
success, the exact match score (per cent correct) gives a geneiral indicator 
of performance on the test item. However, the listing of all readers' 
responses may be more revealing for it shows for the group as a whole how 
much language has been retained (and also how much has been lost), and perhaps 
more importantly the amount of meaning retained (as well as the amount lost) . 

In their analysis of miscues Yetta Goodman and Burke (1972) offer as 
guidelines that losses of 60 per cent or more in language and 70 per cent 
of more in meaning indicate serious weaknesses in students' reading. 
Although these suggestions are for individual students with at least 25 
miscues, it may not be unreasonable, at least until there is sufficient 
evidence to show otherwise, to consider similar guidelines for groups of 
students (say, 25 or more) to individual items as in the present study. 
Assuming such guidelines for a moment, it is noted that language loss (ie. 
grammatically non-acceptable responses) for the 4 passages (Tables 1-4) 
ranges from 5 to 46 per cent, and meaning loss (ie. semantically non- 
acceptable responses) for the same passages ranges from 5 to 56 per cent. 
These losses are within the 60 and 70 per cent cut-off points respectively 
and therefore, if these cut-off points are accepted, do not indicate 
serious weaknesses in students' reading at the points measured. Cettainly, 
examination of the actual responses shows that a majority of students 
(except for the text on Labfador Retrievers, the most difficult of the 
four texts) retain meaning and a clear majority (over all passages) retain 
language . 
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Thus far little has been said about the measure of consensus displayed 
in Tables 1-4. For three of the texts the measure of consensus is virtually 
identical to the exact match score based on the author's original word(s). 
For The Spider text, however, there is a wide divergence between these 
two measures (46 and 12 per cent respectively). It is not difficult to 
see why. Only 6 of the 59 subjects made the same choice as the writer [it]; 
but 6 subjects responded he which is judged semantically and grammatically 
acceptable; a further 15 responded with and , quite acceptable semantically 
and partially acceptable grammatically; a further 5 made the response 
then which again is acceptable semantically and partially acceptable 
grammatically. In other words, 32 subjects made responses that were judged 
semantically and grammatically (fully or partially) acceptable though of 
these only 6 agreed with the writer's actual choice - and it is this 
difference which is largely reflected in the much higher consensus score 
compared with the exact match score. Intuitively, the consensus measure 
seems the more appropriate in this instance. 

As a further illustration of the measures described in this article. 
Tables 5 and 6 display two more text segments, each with three deletions - 
one an instance of lexical reiteration, another of conjunction (adversative 
in one text and temporal in the other), and the third of comparative reference. 
For each deletion there is tabulated the number of different responses 
elicited by the group of 59 subjects, also the most popular response, the 
measure of consensus, and the measure of exact matches. The average 
measure of consensus and of exact matches is also computed over the three 
deletions in each text. As in a previous example, the measure of consensus 
is the higher and examination of the most popular response suggests that 
again it is the more appropriate of the two measures reflecting as it does 
the several ways of completing cohesive ties and of maintaining the semantic 
continuity within texts. Indeed, on occasions, the students' most preferred 
choice, even though these are non-native speakers, seems preferable to 
that of the writer (presumeably an adult, fluent native speaker) . Perhaps 
this ought not to surprise because children, second language speakers 
included, often have more feel for language and write more natural texts 
at their level than those contrived by adults to meet constraints thought 
to be important. 



15 



TABLE 5 



Measures of Consensus and Exact Match, together with Number of Different 
Responses and Most Popular Response, for Fiction T^.xt containing 
Reference, Conjunction and Lexical Cohesion Items (N = 59) 



Mother Duck and the Baby Ducks 

...Down the street trotted Mother Duck, with the five baby ducks close 
behind her... At first they met only a cat... Then they met a dog... Then 
they met a milkman taking round milk and a paper boy. . . The cat stared. 
The stared. The milkman stared. The boy stared. ______ 

Mother Duck trotted on, and the five baby ducks trotted on behind her. 
The morning was not early now. .. 



Item Cohesion No. diff. Most pop. Consensus Exact 

deleted type responses response Match 



dog Lexical 21 dog (23) 41% 39% 

(reiteration) 

but Conjunction 9 the (10) 52% 17% 

(adversative) 

so Reference 16 very (31) 49% 10% 

(comparative) 



Average 47% 22% 
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TABLE 6 



Measures of Consensus and Exact Match, together with Number of 
Different Responses and Most Popular Response, for Non-fiction 
Text containing Reference, Conjunction and Lexical Cohesion 
Items (N = 59) 



Rooks and Mining 

...Sand can be quickly scooped up by a machine called a dredger - ■ 
is used in making glass and concrete. - rocks and minerals are 

mined underground. The miners tunnel under the earth. They blow up 
the rock in the tunnel. they bring out the loose rock. 



Item 
deleted 



Cohesion 
type 



No. diff 
responses 



^fost pop. 
response 



Consensus 



Exact 
Match 



Sand 



Lexical 
(reiteration) 



19 



it (20) 



42% 



17% 



Other 



Then 



Reference 10 
(comparative) 

Conjunction 14 
( temporal ) 



the (32) 
and (25) 



62% 



59% 



0% 



34% 



Average 



54% 



17% 



Surmcay 

within the framework of a general model of communication a model of 
language communication has been developed and applied to the perception 
of cohesion. This model allows an operational definition of the perception 
of textual cohesion and as well brings together established procedures 
from other fields, notably concepts from information theory and theoretical 
insights from miscue analysis. A testing procedure is described which 
requires subjects, if Halliday and Hasan's (1976) text- forming relations, 
are to be perceived, to trace the connection between presupposing and 
presupposed links in text. Some data collected in the U.K. with students 
speaking either Italian or Punjabi as their first language allow the 
methods of measurement to be demonstrated. Although computerised analytical 
methods, termed GAP analysis have been developed (Anderson 1981), the 
methods of analysis are quite easily performed manually. Further, the 
type of analysis exemplified provides potentially useful insights into 
readers' text processing strategies. 
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